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ABSTRACT 

The purpose of this study is twofold: (1) to 

separate variables combined in the forms of Information Feedback 
(IF) ; and (2) to identify additional variables in the effect upon 
retention of 24 hour delay of feedback. In two of the experiments, 
the effect of IF delay, immediate test conditions, and different, 
forms of IF upon seven day retention, measured bv a recall and 
recognition test, were investigated. Undergraduates were subjects and 
learning material was a series of 32 multiple choice items, with a 
definition as a, stem and four uncommon words as alternatives. Results 
revealed that superior retention with delayed IF is due to the fact 
that after a delay subjects acquire more information and when this 
information can be used on the retention test, retention is 
facilitated. Experiment three investigated similar effects but 
subjects* responses to IF were manipulated. It was concluded that the 
way in which subjects respond to IF is an important determiner of 
retention. (Author/EK) 
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Summary 



Recent studies indicate that retention following 24-hr. delay of 
informative feedback (IF) is superior to that following immediate 
presentation of IF, although there is no significant effect upon 
immediate retention, or acquisition. The interpretation of these 
findings is not clear. One possibility is that this effect is due 
to events occurring during the delay interval and that with delayed 
IF the correct sti mu I us- response association is strengthened during 
acquisition, resulting in improved retention. A second interpretation 
is that superior retention with 24-hr. delay of IF is due to factors 
operating at or following the presentation of IF and that what Ss learn 
with immediate and delayed IF actually differs. 

Three experiments investigated the following questions about the 
effect of 24-hr. delay of IF. Does this effect differ with the form 
or IF and, if so, what cues are most likely to be utilized during 
feedback to facilitate retention? Does it depend upon the form of 
retention tests, i.e., recall or recognition? Does it differ with 
the presence and form of an immediate retention test? In all three 
experiments all forms of IF provided the opportunity for the association 
between the stem and the correct alternative. The concern was with 
the effect of additional information presented at feedback. 

Two experiments investigated the effect of delay of IF (0-min., 
20-min., 24-hr.); Immediate test condi tions (nothing; recall; recognition); 
and different forms of IF upon 7-day retention, measured by both a 
recall and recognition test. Undergraduates were Ss; and learning 
material was a series of 32 multiple-choice Items, with a definition 
as a stem and four uncommon English words as alternatives. In 
Experiment I four forms of IF differed in two ways; (a) the number of 
alternatives Included; and (b) the presence or absence of redundant 
cues, not Included on the tests. In Experiment II two forms of IF 
Included the stem with the correct and incorrect alternatives (a) with 
the correct alternative Identified and definitions of each incorrect 
alternative; or (b) with a cue which Ss could use to find the correct 
alternative. Superior retention with delayed IF, on both a recall and 
recognition test, varied with the form of IF and the immediate test 
condition. Following an Immediate recognition test, retention was 
superior with delayed feedback when IF was In variable form but not 
when It was redundant. When IF was a cue directing the Ss 1 attention 
to all alternatives, retention was superior with delayed feedback 
only with no immediate test. When IF was a cue, following an 
immediate test retention for Ss with immediate feedback improved and 
delayed feedback was no longer superior. These findings were 
interpreted as supporting the hypothesis that superior retention with 
delayed IF Is due to the fact that after a delay interval Ss acquire 
more information at IF and that when this Information can be used on 
the retention test, retention is facilitated. 

Experiment III investigated the question; can effects similar 
to those of delayed IF be obtained by manipulating Ss 1 responses to 
IF. The Ss reactions to IF were manipulated by the form of IF and 
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instructions and were designed to compare the effect of: (a) learning 

the correct alternative only; (b) learning the correct and the incorrect 
alternatives; and (c) organizing the material. Except for the feedback 
conditions, the design and materials were identical to those of 
Experiments I and II, Following an immediate test, retention on a 
recall test was superior when Ss had been instructed to study the 
incorrect in addition to the correct alternative. On the recognition 
test, retention was superior when Ss had been instructed to study the 
correct alternative only. These results were interpreted as supporting 
the hypothesis that how J>s respond to IF is an important determiner of 
their retention of the material. Also, they were seen as supporting 
the Interpretation that retention with minimal cues is facilitated 
when Ss have acquired information in addition to the correct 
stimulus-response members. 
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Introduction 

Many educational practices incorporate the principle of the 
superiority of immediate informative feedback (IF) both for initial 
learning and for retention. These practices include most programs for 
teaching machines and programmed texts, but they are also employed in 
classroom activities such as returning tests, replying to students* 
questions, daily assignments and, in general, imparting information in 
the classroom. Recent studies question the general superiority of 
immediate IF with meaningful verbal material. Experimental findings 
indicate that there is no detrimental effect upon acquisition with a 
delay of IF unless the delay interferes with conditions necessary for 
association of the stimulus and the correct response alternative, for 
example, the association of the question with the correct answer 
(I, 2, 7, 10). Thus a number of studies in which IF includes the 
entire item have found no difference in measures of acquisition when IF 
was presented immediately or after a short (10-sec.) or longer (24-hr.) 
delay interval (3, 4, 5, 6, 13, 15, 12). 

More interesting are the findings that under some conditions 
retention Is superior when IF is delayed rather than presented 
immediately after the response. Brackbi 1 1 and her associates 
(3, 4, 5, 6) have consistently found superior retention both one and 
seven days after acquisition when a I 0-sec. delay of feedback was used 
during acquisition. These studies have used children as Ss and a 
variety of two-choice discrimination tasks, including pictures of 
familiar objects, nonsense bi- 2 r ams, meaningful words, and English-French 
equivalents. In general, they have found that the delay retention 
effect is greater with more complex materials. In Brackbi 1 1 *s studies 
feedback consisted of the combination of tangible reinforcement (marbles 
which were exchanged for a toy); knowledge of results (a click or a 
buzz if the S*s response had been correct or incorrect respectively); 
and informative feedback (a light over the correct response). 

Markowitz and Renner (10) investigated the delay retention effect by 
separating the tangible reinforcement from the combined auditory signal 
and IF used by Brackbi II. With tangible reinforcement only, they found 
the delay retention effect when feedback was the combination used by 
Brackbi 1 1 and also when Ss received immediate tangible reinforcement 
and delayed auditory signal and IF. Markowitz and Rennar concluded 
that the delay retention effect was due to the unusual combination of 
feedback used by Brackbi II. They suggested that when this combined 
feedback was delayed, Ss had to look at the light to determine the 
correct response and that watching the light Increased the memory of 
the correct alternative; whereas with immediate feedback, Ss did not 
have to look at the light to determine the correct response. Sturges, 
Sarafino, and Donaldson (15) Investigated the delay retention effect 
with a procedure similar to that used by Brackbi 1 1 and by Markowitz and 
Renner, except that feedback consisted only of IF, the light over the 
correct alternative. Their findings were similar to those of Brackbi 1 1 
and of Markowitz and Renner; superior savings were shown at relearning 
by Ss receiving delayed IF during acquisition. This finding of the 
delay retention effect when the only feedback was IF questions the 
conclusions of Markowitz and Renner. 

Llntz and Brackbi 1 1 (8) investigated the delay retention effect 
with adults with conditions equivalent to those with children and 
found equivocal results. With nonsense bl -grams presented In a 
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two-choice discrimination task, there was no difference in retention 
between immediate and delayed feedback during acquisition. The 
delay retention effect was found when the same bi -grams were 
presented in a paired-associate task, 

Brackbi 1 1 (3) investigated her hypothesis that the delay 
retention effect occurred because more task-related covert responses 
were made during the I 0-sec. delay interval, were subsequently 
reinforced, and thus were available at retention. An extraneous task 
was introduced during the I 0-sec, interval preceding feedback for Ss 
in the delay group; the same task was introduced following feedback for 
Ss receiving immediate feedback. There was no effect of the extraneous 
task upon either acquisition or retention for Sjs with delayed feedback. 
When the task was introduced right after immediate feedback, however, 
a decrement both at acquisition and at retention resulted. This 
finding suggests that the delay retention effect is due to factors 
operating during or immediately following IF rather than to factors 
occurring during the delay interval. 

Recently, a few studies have investigated the effect of a 24-hr. 
delay of IF, with only one acquisition trial p i.e., presentation of 
IF, and the major dependent variable a retention measure. Thus, Ss 
have been presented a series of items and made an initial "guessing” 
response to each; and then 24-hr. later they have been presented the 
series of informative feedback. This 24-hr. delay condition has been 
compared either with one in which feedback is presented immediately 
item-by-item (13) or with a condition in which the series of feedback 
is presented immediately after the series of items (12). in some 
studies Ss have been tested immediately for acquisition (II, 12, 13) 
and in all they have been tested several days later for retention. 

The general results of these investigations have been that at 
acquisition there is no significant difference between delayed and 
immediate feedback but on later retention delayed feedback is superior. 

Although the effects of a 24-hr. delay of feedback have some 
similarity to those with a shorter delay interval, these investigations 
differ in several important ways, and there is reason to question that 
they can be directly compared. However, the effects of a 24-hr. 
delay interval are important for at least two reasons. First, the 
operations are those of a delay of IF and thus the results offer 
information on the generality of findings with shorter delay intervals. 
Second, the effect of the longer delay interval upon retention is 
important in understanding factors involved in long-term retention. 

Sturges and Crawford (14) investigated the effect upon 7-day 
retention of a 24-hr. delay of IF, with college students as Ss. 

They found that the delay retention effect was a function of the 
meaningful ness of material and the type of IF. When feedback indicated 
the correct answer, the delay retention effect was found with factual 
items and with material requiring inductive generalizations. There 
was no difference in retention between immediate and delayed IF for 
nonsense materials when IF was the correct answer or with meaningful 
material when IF was only a cue suggesting the correct answer. Their 
interpretation was that exploration of meaningful material, prior to 
or at feedback, facilitates retention. They suggested that such 
exploration was encouraged by delaying the presentation of the correct 
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alternative or by presenting a cue which would directly promote such 
exploration of the material. 

Sassenrath and Yonge (12) Investigated the effect upon 5-day 
retention with college students as Ss of a 24-hr. delay of IF, presence 
or absence of the stem at IF, and a set to remember. Their results 
indicated superior retention with delayed IF, the stem of the question, 
and the retention set, but there were no significant interactions 
among these variables. More (II) compared 7-day retention with 
Junior High School students following Immediate IF, 2-hr,, 24-hr,, 
and 4-day delay of IF and found superior retention with 24-hr. delay 
of IF. 



An Important question in Investigating factors that lead to 
Improved retention with 24-hr. delay of IF is: what are Ss actually 

learning with Immediate and delayed IF? One possibility is that this 
effect Is due to events occurring during the delay Interval and that 
In both conditions Ss are learning primarily or solely the correct 
stimulus-response association. Thus, with delayed IF the correct 
members would be strengthened during acquisition, resulting In Improved 
retention. With this hypothesis the superiority of delayed IF should 
occur on immediate retention or acquisition, which has not been found. 
However, It may be that Ss In both conditions learn the same thing, 
and Initially to about the same level, but that during the delay 
Interval a process occurs that is similar to the Zelgarnik effect 
of an Incomplete task, making the correct stimulus-response 
association a more salient aspect and thus resulting In better 
retention. 

A second possibility is that the difference In retention with 
Immediate and 24-hr. delay of feedback Is due to factors operating 
at or following the presentation of IF and that what Ss learn with 
immediate and delayed IF actually differs. It may be that with 
delayed IF Ss respond to more cues or stimulus aspects of IF, thus 
learning more about the item; and that when these cues can be used 
In retention, delay improves retention. This hypothesis suggests that 
Ss respond differently to IF when it Is presented Immediately after 
the response than they do when It Is presented after a delay Interval; 
and that the way they respond to the IF determines what they learn 
and, therefore, their retention of the material. With this hypothesis 
there are also two possibilities. One Is that the additional learning 
postulated to occur with delayed feedback results In more precise 
discrimination of the correct choice due to S^ having learned both the 
Incorrect and the correct alternatives. This Interpretation would be 
similar to concept Identification In which Ss learn to identify the 
negative as well as the positive Instances of the concept. A second 
possibility with this hypothesis is that the additional learning 
postulated with delayed IF results In higher-order organization of the 
Items. Recent work by Mandler (9) indicates that retention of verbal 
material is a function of the organization of the malerial, i.e., 
superior recall Is found when Ss have identified stable relationships 
among the verbal units. In his studies, Mandler measured "free recall" 
of Individual words. However, It may be that in verbal learning tasks 
in which £ must learn to discriminate the correct from the incorrect 
alternatives, retention Is improved as he explores and organizes the 
learning material more completely. According to this interpretation. 
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the effect of delayed feedback would depend upon: (a) stimulus 

aspects present during feedback; and (b) the relevance of these stimuli 
to the retention test. 

Some support for this second Interpretation was found In an 
earlier experiment (13). Superior retention with 24-hr. delay of 
Informative feedback occurred when feedback Included the Incorrect In 
addition to the correct alternative but not when It Included the correct 
alternative only. Thus, these findings support the hypothesis that the 
effect of 24-hr. delay of feedback Is due to factors operating at or 
following the presentation of feedback rather than to events occurring 
during the delay Interval. However, the Inclusion of Incorrect 
alternatives was confounded with the presence of spatial cues and 
letters of the alternatives, both of which were directly relevant to 
the retention tests. Thus, it Is not clear whether these results were 
due to the redundancy of feedback cues, to the utilization of more cues 
in general, and/or to the knowledge of the specific alternatives. 

Three experiments were conducted to provide a more thorough 
understanding of variables Involved In the delay retention effect. One 
purpose was to separate the variables combined In the forms of IF In 
Sturges* (13) experiment. A second purpose was to Identify some 
additional variables Involved In the effect of 24-hr. delay of feedback 
upon retention. Thus, the experiments Investigated the following 
questions about the effect of 24-hr. delay of feedback. Does this 
effect differ with the form of feedback and. If so, what cues are most 
likely to be utilized during feedback to facilitate retention? Does 
It depend upon the form of retention tests, i.e., recall or 
recognition? Does It differ with the presence and form of an 
Immediate retention test? In all three experiments all forms of 
feedback provided the opportunity for the association between the stem 
and the correct alternative. The concern was with the effect of 
additional Information presented at feedback. 

Experiment I 

Experiment I Investigated the effect of four forms of feedback, 
which differed In two ways: (a) the number of alternatives Included; 

and (b) the presence or absence of redundant cues, position and the 
letters of the alternatives. The redundant stimuli were not included 
on either the immediate or 7-day recognition test and thus were not 
directly relevant to the tests. 

Three delay intervals were compared: 0-min. delay. In which 

feedback was presented Immediately Item-by-ltem; 20-ml n. delay. In 
which the series of feedback was presented Immediately after the 
series of Items; and 24-hr. delay. In which Ss received the series 
of items on the first day and returned 24 hours later for the series 
of feedback. Previous studies have found superior retention for 
24-hr. delay compared with both a condition In which feedback was 
presented immediately Item-by-ltem (13, 14) and with a condition In 
which the series of feedback was presented Immediately after the 
series of Items ( 52 ). This latter condition is not Immediate IF in 
the sense of feedback following Immediately upon the response but, 
rather. It Is a 20-ml n. experimentally filled delay interval. However, 
In the 20-mi n. delay condition the sequence of experimental events Is 
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the same as In the 24-hr. delay, while In the O-mln. delay, the 
sequence Is different. Thus, these three delay Intervals permit 
evaluation of the effect of the sequence of experimental events 
(0-mln. ) and experimental activity during the delay Interval (20-min.) 
In addition to the length of delay. 

' etent I on was measured by both a recall and a recognition test 
to provide additional Information about what Ss are learning with 
different delay Intervals and different forms of feedback. If the 
delay retention effect Is found only with a recognition test. It 
would seem to be dependent upon presentation of the entire item on 
the test. Thus, it may be due to minimal learning, to discrimination 
among alternatives on a recognition level, etc. However, if the 
delay retention effect Is a result of higher-order organization of 
the material, then It should be found with a recall test as well as 
with a recognition test. That Is, If Ss have actively organized the 
material by Identifying relations among the stem, the correct, and 
the Incorrect alternatives, they should then be able to recall It with 
minimal cues at retention. 

< 

The fourth variable Investigated Is primarily of methodological 
Interest, but It also has Implications for explanation of the delay 
retention effect. Both Sturges ( 13) and Sassenrath and Yonge ( 12) 
Included an Immediate and a 7-day retention test on the same Ss. 

Thus, Ss In both immediate and delayed IF groups had an additional 
presentation of the material following IF (the immediate retention 
test), thereby confounding the measure of 7-day retention. It would 
be expected that later retention In both groups would be improved 
with this additional presentation of the material, counteracting to 
some extent the effect of factors operating at acquisition. 
Furthermore, it would be predicted that the additional presentation 
would decrease the delay retention effect, if this effect is due to 
the fact that Ss with delayed IF explore the material at the time 
feedback is presented while Ss with immediate IF do not. That is, 
it would be likely that Ss In both groups would explore the material 
at or following the presentation of the immediate retention test. 

Thus, any superiority of retention due to additional exploration of 
the material by the delay group at the time of IF would be reduced. 
The present study investigated three immediate test conditions. 
Subjects received either no immediate test, an Immediate recall test, 
or an immediate recognition test; and all Ss were given both a 
recall and a recognition test at seven days. This provided a 
measure of immediate retention, or acquisition, and also evaluation 
of the effects of immediate tests upon the 7-day retention tests. 

Method 

Design. Three variables were combined factor la My: four 

forms of feedback [Right-Wrong Redundant (RW+); Right Redundant 
(R+); Right-Wrong Variable (RW); Right Variable (R)]; three delay 
intervals (0-min., 20-min., 24-hr.); and three immediate test 
conditions [nothing, recall (Re-1), recognition (Rcg-|)] All Ss 
had two 7-day retention tests, recall and recognition. Subjects 
were 468 undergraduates, fulfilling a course requirement, who were 
randomly assigned with 13 Ss in each of the 36 groups. 




Apparatus and learning material . Learning material was a 
series of 32 multiple-choice items, with a definition as a stem and 
four uncommon English words as alternatives* The task was to learn 
the correct word for each definition* The items were selected with 
eight from each of four word categories: concrete nouns; abstract 

nouns; adjectives; and verbs* For the initial presentation, each 
item consisted of the stem with each alternative below it and 
preceded by a letter (a, b, c, d). Each form of feedback included 
the stem and the correct alternative, which was underlined and had 
an asterisk to its left* Two forms of feedback had the stem and the 
correct alternative only (R, R+); and two had the stem with the correct 
and all incorrect alternatives (RW, RW+), For each of these one was 
redundant, with the letters and each alternative in the same position 
as in the initial presentation. For the two variable forms of 
feedback, each alternative was In a randomly different position and 
without the letters. See Appendix A for symple item and different 
forms of feedback* 

The items were presented in the same random order for the 
initial and IF presentations. On both immediate and 7-day tests the 
items were in different random orders, and on the recognition tests 
the alternatives were in randomly different positions with no letters 
preceding them. The recognition test presented the stem and all four 
alternatives; the recall test presented the stem only; and in both 
tests Ss wrote the correct alternative. 

All material was presented on 2 in. X 2 in. slides by a Kodak 
Carousel slide projector, with presentation intervals automatically 
controlled by electronic timing units. For the initial presentation 
and the retention tests Ss recorded their answers on special devices, 
designed so that the answer was turned out of view immediately. 

Procedure. Subjects in all groups participated In the 
following three phases of the experiment: initial presentation of 

the material with the Ss answering each item; presentation of IF; 
and both 7-day retention tests. For Ss with 0-mln. and 20-mi n. 
delay of IF, the initial presentation of the material and the 
presentation of IF occurred in the same session. For the 0-min. 
groups the sequence for this initial session was: item I (15 sec.); 

write the answer (15 sec.); IF for item I (15 sec.); rest (10 sec.); 

item 2 (15 sec.); etc. For the 20-min. delay groups the series of 

items was followed immediately by the series of IF: item I (15 sec.); 
write the answer (15 sec.); item 2 (15 sec.). • • item 32 (15 sec.); 

write the answer (15 sec.); IF for item I (15 sec.); rest (10 sec.); 

etc. For the 24-hr. delay groups the sequence of events was the 
same as for the 20-min. delay groups except that Ss received the 
series of items on the first day and returned 24 hours later for 
the series of IF. In all three delay conditions, Ss had the same 
number, type, and length of presentation of the initial material 
and of IF. For Ss in the immediate test group, this retention test 
(Re-1; Rcg-I), was given immediately after the series of IF: on 
Session I for the 0- and 20-min. delay groups; and on Session 2 for 
the 24-hr. delay groups. All Ss were given a recall test followed 
by a recognition test seven days after the presentation of iF. 

In both the recognition test seven days after the presentation 
of IF. In both the recognition and the recall tests, the temporal 
sequence for all groups was: item I (15 sec.); write the answer 
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(15 sec.); Item 2 (15 sec,); etc. See Appendix B for the temporal 
sequence of experimental events, 

Resu I ts 

Immediate Tests . Figure I presents the mean correct for each 
group for each of the Immediate tests. Table I presents the summary 
of the analysis of variance of these data. The overall effect of 

Table I 

Summary Analysis of Variance, Immediate Tests, Experiment I 









Source 


SS 


df 


MS 


F 



Feedback (F) 






29.52 


3 


9.84 


1.68 


RW+ > RW; R > 


R+ 


CM 

U~ 

X 

uu 


28.32 


1 


28,32 


4.84* 


Delay (D) 






168.51 


2 


84,25 


14.40*** 


(24- + 20- ) > 


0- 


(D, > 


160.03 


1 


160.03 


27.47*** 


Test (T) 






6526.70 


1 


6526.70 


1 1 1 5.68*** 


F x D 






38.30 


6 


6.38 


1.09 


F x T 






7,40 


3 


2.47 


.42 


D x T 






41.97 


2 


20.98 


3.59* 


D, xT 






41.03 


1 


41.03 


7.02** 


F x D x T 






31.69 


6 


5.28 


.90 


error 






1683.73 


288 


5.85 





*EL < *05 
**£ < .01 
***£< .001 



NOTE: 

The overall effect of delay was divided Into two orthogonal components: 
D. (24-hr. + 20-mln.) vs. 0-mln. 

D 2 24-hr. vs. 20-mln. 

The overall effect of form of feedback was divided Into three 
orthogonal components: 

F, RW (RW+, RW) vs. R (R+, R) 

F 2 Variable (RW, R) vs. Redundant (RW+, R+) 



The overall effect of Immediate test condition (on 7-day retention) 
was divided Into two orthogonal components: 

Tj The combined tests (Re- I + Rcg-I) vs. no test 
T 2 Re- I vs. Rcg-I 
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delay was divided Into two orthogonal components: Dj, (24-hr. + 

20-mln.) vs. 0-mln.; and D 2 , 24-hr. vs. 20-mln. The effect of form of 
IF (F) was divided into three orthogonal components: Fj, Right Wrong 

(RW+, RW) vs. Right only (R+, R); F 2 , variable (RW, R) vs. redundant 
(RW+, R+); and the interaction of these two components. 



The combined delay groups (24-hr. + 20-mln.) were superior to 
0-mln. delay, and this effect was more marked on the recall test than 
on the recognition test. These results are contrary to those of other 
studies, but this difference can be attributed to the fact that the 
effect of delay was found primarily on the recall test and for the 
combined 24-hr. and 20-min. delay groups only. In previous studies 
reporting no effect of delay on immediate retention, either a 
recognition test has been used or a 24-hr. delay was compared with 
20-mln. delay. 

There was a significant interaction between the forms of 
feedback. When all alternatives were presented at IF redundant was 
superior to variable but when the correct alternative only was 
presented, variable was superior to redundant. Also, performance was 
significantly greater on the recognition test than on the recaTt hast. 



Seven-day retention tests. Figures 2, 3, and 4 present the 
mean correct for each of the groups and each of the 7-day retention 
tests. Table 2 presents the summary of the analysis of variance of 
these data. The same orthogonal comparisons were made as for the 
immediate tests. Also, the effect of the immediate test condition 
was divided into two components: Tj, the combined tests (Re- I + Rcg-I) 

vs. no test; and T 2 , Re-1 vs. Rcg-I. 

There was a significant overall effect of delay, with both 
components significant. The combined 24-hr. and 20-min. delay was 
superior to 20-min. delay. This latter finding is consistent with 
that of Sassenrath and Yonge (12) in which they compared 24-hr. 
delay with an immediate group similar to the 20-min. delay in the 
present study. 

As on the immediate tests, the superiority of the combined delay 
groups was greater on the recall test than on the recognition test. 
Also, overall performance on the recognition test was significantly 
better than on the recall test. 



There was an overall significant effect of the immediate test 
condition, with both components significant. Seven-day retention 
for the combined immediate test groups was superior to that with no 
immediate test; and retention following an immediate recognition test 
was superior to that with an immediate recall test. There was also 
a significant interaction between immediate test and form of 7-day 
test (T X R). The superior retention following the immediate 
recognition test (Rcg-I) was greater on the 7-day recognition test 
than on the 7-day recall test (see Figures 3 and 4). 

There was no overall effect of form of feedback, but there were 
some significant interaction effects between delay, form of feedback, 
and the other variables. One component of the interaction of form of 
feedback and type of retention test (F X R) was significant. 
Performance on the recall test was best when IF included all 
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Table 2 



Summary Analysis of Variance, 7-day Retention Tests, Experiment I 





Source 


SS 


df 


MS 


F 



Feedback (F) 


48.92 


3 


16.31 


1.70 


(R + R+) > (RW + RW+) 


<F,> 


1 


31.98 


3.33 


Delay (D) 


387.15 


2 


193.57 


20. 14*** 


(24- + 20) > 0- (D, ) 




1 


292.13 


30.40*** 


24- > 20- (D 2 ) 




1 


95.02 


9.89** 


Test, Immediate (T) 


682.90 


2 


341.45 


## 


(Re- 1 + Reg- 1 ) > none 


(T | ) 559.04 
123.85 


1 


559.04 


57.69*** 


Rcg-I > Re-1 (T 2 > 


1 


123.85 


12.89*** 


F x D 


27.36 


6 


4.56 


.47 


F x T 


26.19 


6 


4.36 


.45 


DxT 


61.14 


4 


15.29 


1,59 


0, x T, 


30.66 


1 


30.66 


3.19 


F x D x T 


130.42 


12 


10.87 


1.13 


Oj x F2 x T 2 




1 


54.17 


5.64* 


error (between) 


4151.27 


432 


9.61 




Retention Test (R) 


15,794,90 


1 


15,794.90 


9515.00*** 


F x R 


11.63 


3 


3.88 


2.33 


F | x R 




1 


8,46 


6.00* 


D x R 


9.31 


2 


4.66 


2.80 


D| x R 




1 


8.82 


5.31* 


T x R 


98.16 


2 


49.08 


29,56*** 


T 2 x R 




1 


96.98 


58.41*** 


F x D x R 


13.14 


6 


2.19 


1.32 


D 2 x (F, x F 2 ) x R 




1 


6.88 


4.14* 


F x T x R 


12.19 


6 


2.03 


1.22 


D x T x R 


2.48 


4 


0.62 


.37 


D x F x T x R 


13.47 


12 


1.12 


.67 


error (within) 


717.23 


432 


1.66 





*£ < .05 
**£< .01 
***£ < .001 
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^ ^ Right-Wrong Redundant (RWt) 

' ’ — O Right Redundant (R+) 

G — — Right-Wrong Variable (RW) 

O X Right Variable (R) 




Delay Interval 
A. Recall Test 




Delay Interval 

B„ Recognition Test 
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Fig. 2 Mean Correct, 7-day Retention Tests, Experiment I 
with Immediate Test Conditions Combined, for Delay 
Conditions and Four Forms of Feedback 
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alternatives (RW, RW+), and on the recognition it was best for IF with 
the correct alternative only (R, R+). 

One component of the interaction of form of IF, delay, and 
immediate test (F X D X T) was significant. The superiority of the 
combined 24-hr. and 20-mi n. delay groups differed with the form of 
feedback and the immediate test conditions. This is most readily seen 
in Figure 4. Following an immediate recognition test there is a marked 
relationship between delay and variable-redundant form of IF. Superior 
retention with delay occurred when feedback was in variable form but 
not when it was redundant. Following an immediate recall test the 
relationship between delay for form of IF is quite different. The 
slight superiority of delay with redundant feedback is due almost 
solely to the inferiority of one group, Right-Wrong Variable, 20-min. 
delay. 



One component of the interaction between form of IF, delay, and 
form of retention test (F X D X R) was also significant. The 
superiority of 24-hr. delay to 20-mln. delay was a function of the type 
of retention test and the interaction of RW-R and variable-redundant 
forms of feedback. As can be seen in Figure 2, this effect is 
accounted for primari ly by the greater superiority on the 7-day recall 
test with Right-Wrong Variable (RW) than with Right-Wrong Redundant 
(RW+) and with Right Redundant (R+) than with Right Variable (R). 

The opposite relationship occurred on the 7-day recognition test, 
although to a lesser degree. 

Discussion 

According to these results, the superiority of the combined 
delay conditions did differ with the form of feedback and the 
immediate test conditions. These findings are consistent with the 
prediction that the effect of delayed feedback depends upon the 
stimulus aspects present during feedback and the relevance of these 
stimuli to the retention test. With redundant feedback, where the 
additional information was not relevant to the test, retention did not 
vary with delay. With variable forms of feedback, there was a marked 
superiority of delayed feedback. These results are consistent with 
those of the earlier study (13) in that in both studies the delay 
retention effect was found when IF presented stimuli relevant to both 
an Immediate and 7-day test. However, in the present study the delay 
retention effect occurred for both variable forms of IF, that with 
all alternatives and that with the correct alternative only. Thus, it 
would seem that the delay retention effect does not depend upon the 
presentation of the Incorrect alternatives per se but rather upon the 
number of relevant cues presented at IF. 

The fact that the relationship between the delay retention 
effect and variable-redundant forms of IF was found with both 7-day 
retention measures indicates that the effect is not dependent upon the 
form of the test at 7 days. That is, there is no evidence that 
presentation of the entire item on the 7-day test either facilitated 
or Interfered with this effect. However, the form of the immediate 
test did make a difference. Apparently, the differential effect of 
delay with different forms of feedback depends upon the relationship 
between information presen+ed at feedback and on the immediate test. 



Experiment 1 1 



Experiment II investigated the effect of delay as a function of 
two different forms of feedback. One form of feedback was selected to 
provide more information on what cues are utilized to facilitate 
retention and presented more additional information, the definition for 
each Incorrect alternative. This condition contrasted with both 
Right Wrong-Variable and Right Wrong-Redundant in Experiment I. The 
additional Information here is neither the same as in the initial 
presentation nor directly relevant to the recognition tests. 

The second form of feedback provided a more direct test of the 
hypothesis that superior retention with delay occurs because Ss respond 
to all information present at feedback after a delay interval but not 
when feedback is immediate. In this form of feedback the entire item 
was presented, the correct alternative was not indicated, but a cue 
was included which the could use to find the correct alternative. 

With this form of feedback Ss in both immediate and deHay conditions 
should be directed to explore all alternatives. Thus, if this is the 
factor producing the delay retention effect, the superiority of delay 
should disappear. 

Method 



Des i gn . Three variables were combined factorial ly; two forms of 
feedback [Right Wrong Definitions (RW-D); and Right Wrong Cue (RW-C);]; 
three delay intervals (0-min., 20-min., and 24-hr.); and three 
Immediate test conditions [nothing, recall (Re-1), recognition (Rcg-I)]. 
All Ss had both 7-day retention tests, recall and recognition. 

Subjects were 180 undergraduates fulfilling a course requirement, 
randomly assigned with 10 Ss in each of the 18 groups. 

Learning material and procedure. All material and procedures 
were Identical to Experiment I except for the form of feedback and the 
length of presentation of IF. For feedback consisting of Right Wrong 
Definitions, the entire item was presented, the correct alternative 
was underlined and asterisked, and a definition was Included after each 
Incorrect alternative. For feedback consisting of Right Wrong Cue the 
entire item was presented, the correct alternative was not indicated, 
but a cue was included which $_ could use to find the correct alternative. 
For example, for the item where the stem was TO REPEAT SENSELESSLY 
and the correct alternative was VERBIGERATE, the cue was VERBUM = 

WORD AS IN VERB; -ATE = TO MAKE. In both forms of feedback the 
alternatives were In randomly different positions without letters and 
thus were also variable. For all groups IF was presented for 18 sec. 

All other temporal Intervals were the same as in Experiment I. 

Results 



Immediate tests . Figure 5 presents the mean correct for each 
group for each of the immediate tests. Table 3 presents the summary 
of the analysis of variance of these data. As in Experiment I the 
overall effect of delay was divided Into two orthogonal components: 
Dj (24-hr. + 20-min.) vs. 0-min.; and D 2 24-hr. vs. 20-min. 
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# • Right-Wrong Definition (RW-D) 

O O Right-Wrong Cue (RW-C) 
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B. Recognition Test 



Fig. 5 Mean Correct, Immediate Retention Tests, Experiment 
for Delay Conditions and Two Forms of Feedback 



Table 3 



Summary Analysis of Variance, Immediate Tests, Experiment II 



Source 


SS 


df 


MS 


F 


Feedback (F) 


0.21 


1 


0.21 


< 1 


Delay (D) 


8.45 


2 


4.23 


< 1 


Test (T) 


2475.21 


1 


2475.21 


405.77*** 


F x D 


1.08 


2 


0.54 


< 1 


F x T 


0.03 


1 


0.03 


< 1 


D x T 


4.88 


2 


2.44 


< 1 


F x D x T 


1.20 


2 


0.60 


< 1 


error 


659.10 


108 


6.10 





***£. < .001 



The only significant effect was that performance on the 
recognition test was superior to that on the recall test. There was no 
reliable effect of delay or form of feedback. 

Seven-day tests . Figures 6, 7, and 8 present the mean correct 
for each group for each of the tests. Table 4 presents the summary of 
the analysis of variance of these data. As in Experiment I, the 
effect of the immediate test condition was divided into two components; 
Tj, the combined tests (Re-1 + Rcg-I) vs. no test; and To, Re-1 vs. 

Reg- I . 



Overall retention for the combined 24-hr. and 20-mi n. delay 
groups was significantly greater than that for 0-min. delay; but there 
was no reliable overall difference between 24-hr. and 20-min. delay. 
The overall effects of the immediate test conditions and the type of 
retention test were the same as in Experiment I. Retention was 
significantly better on the recognition test than on the recall test 
and also it was superior following the combined immediate tests to 
that with no immediate test. Following the immediate recognition 
test retention was better than following the immediate recall test; 
and this effect was significantly greater on the recognition test 
than on the reca 1 1 test. 

Also, as in Experiment I, there was no overall effect of the 
form of feedback but there were some significant interaction effects 
between delay, form of feedback, and the other variables. One 
component of the interaction between form of IF, delay and immediate 
test condition (F x D x T) was significant. The effect of the 
immediate tests upon the superiority of the combined 24-hr. and 20-min. 
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B. Recognition Test 



Fig. 6 Mean Correct, 7-day Retention Tests, Experiment I I 
with Immediate Test Conditions Combined, for Delay 
Conditions and Two Forms of Feedback 
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Table 4 



Summary Analysis of Variance, 7-day Retention Tests, Experiment II 



Source 


SS 


df 


MS 


F 


Feedback (F) 


26.95 


1 


26.95 


2.98 


Delay (D) 


78.76 


2 


39.38 


4.36* 


(24- + 20- ) >0 (D j ) 


66.61 


1 


66.61 


7.37** 


Tests, Immediate (T) 


273.68 


2 


136.84 


15.15*** 


(Rcg-I + Re-1) > none (T|) 


193.75 


1 


193.75 


21.44*** 


Rcg-I > Re-1 (T 2 ) 


79.93 


1 


79.93 


8.85** 


F x D 


7.18 


2 


3.59 


.39 


F x T 


25.85 


2 


12.93 


1.43 


D x T 


31.62 


4 


7.91 


GO 

CO 

• 


F x D x T 


74.07 


4 


18.52 


2.05 


D| x Tj x F 


62.71 


1 


62.71 


6.94** 


error (between) 


1463.71 


162 


9.04 




Retention Test (R) 


6138.01 


1 


6138.01 


4212.19*** 


F x R 


1.81 


1 


1.81 


1.24 


0 x R 


2.96 


2 


1.48 


1.02 


T x R 


42.99 


2 


21.49 


14.75*** 


T 2 x R 


42.08 


1 


42.08 


28.88*** 


F x D x R 


4.80 


2 


2.40 


1.65 


F x T x R 


0.79 


2 


0.39 


.27 


D x T x R 


5.34 


4 


1.33 


.91 


F x D x T x R 


15.87 


4 


3.97 


2.72 


D 2 x To x F x R 


7.01 


1 


7.01 


4.81* 


Dj x Tj x F x R 


7.44 


1 


7.44 


5.11* 


error (within) 


236.06 


162 


1.46 





*p < .05 

< .01 

***p < .001 
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delay groups was essentially the opposite for the two forms of IF. 
When feedback was a cue (RW-C), the delay groups were superior 
only when there was no Immediate test. Following an immediate 
test, when IF was a cue, 7-day retention with 0-min. delay 
improved and delayed feedback was no longer superior. In fact, 
under these conditions, 7-day retention with 0-min. delay did 
not differ appreciably from that for the delay groups with any 
form of feedback in either Experiment I or II. 

When feedback identified the correct alternative and 
presented definitions of the incorrect (RW-D), delay was superior 
following the two immediate tests but not with no immediate test. 

This finding was similar to that In Experiment I for variable 
feedback, indicating that this effect does not require that the 
additional information be directly relevant to the test. One 
significant component of the four-way interaction between delay, 
form of IF, immediate test condition, and form of retention test 
(D x F x T x R) indicated that this latter effect with RW-D was 
greater on the 7-day recall test than on the recognition test. 

A second significant component of the interaction between delay, 
form of feedback, immediate test conditions, and form of retention 
test (D x F x T x R) indicated that the relative superiority of 
24-hr. to 20-mi n. delay differed with the two forms of IF, and 
the form of both the immediate and 7-day tests. 

Discussion 

The results of Experiments I and II indicate that the effect 
of delayed feedback upon retention does differ with the form of 
feedback, and that this effect Is dependent upon what happens 
immediately after acquisition, I.e., the immediate test condition. 
Which stimulus characteristics present at feedback are utilized to 
facilitate retention also depends upon both the Immediate test 
condition and the delay interval. These results are consistent 
with the interpretation that after a delay interval Ss learn more 
at feedback, and that the retention of what is learned is affected 
by the immediate test. 

What are Ss learning with delayed feedback that facilitates 
retention? In the different forms of feedback, different information 
in addition to the correct stimulus-response words was presented. 
However, the effect of this additional information may have been 
primarily to make more salient the correct alternative. In 
Experiment I there was no differential effect of delay for the 
two variable forms of IF, the correct alternative only or the 
correct and incorrect alternatives. Some evidence on the question 
of what is learned is provided by the kind of errors made on the 
recall test. The percentage of errors that were incorrect 
alternatives from the same item was computed. Figure 9 shows 
these data for the 7-day recall test in Experiment I. Again the 
immediate test conditions made a difference. Following an 
immediate recognition test there is a marked relationship between 
the percentage of errors of this kind, the delay interval, and 
form of IF. With 24-hr. delay there was an increase in percentage 
of errors that were incorrect alternatives when IF had included 
all alternatives. With 0-min. delay there was essentially no 
relationship between the kinds of errors and form of IF. These 
findings offer support for the interpretation that after a delay 
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interval Ss acquire more information presented at IF and that the 
retention of what is learned is affected by the immediate test. 

Two retention tests were used to provide evidence on what 
Ss are learning with delay of IF. Superior retention with delay, 
both overall and as a function of the form of IF, occurred on 
both a recall and recognition test, and in some conditions it was 
significantly greater on the recall test. Thus, the effect of 
delay is not due solely to improved discrimination among 
alternatives at a recognition level or to minimal retention, either 
of which would require the entire item to be presented. Rather, 
whatever is learned with delay is available with minimal cues at 
retention. The recall test was included as a measure more sensitive 
to organization of or among the alternatives, and the present 
findings provide some support for this assumption. In Experiment I 
overall performance on the 7-day recall test was best when IF 
consisted of all alternatives and on the 7-day recognition test 
when it was the correct alternative only. Thus, recall of the 
correct alternative to the stem word was facilitated when Ss had 
an opportunity to learn the incorrect in addition to the correct 
alternative. These findings suggest that retention of the correct 
stimulus-response words is facilitated when Sjs have identified 
relations among the stem, the correct, and the incorrect 
alternatives. 



Experiment H provides additional support for the hypothesis 
that superior retention with delayed IF is due to the information 
acquired at IF. When IF presented a cue which could be used to 
find the correct alternative, retention for Ss with immediate 
feedback was equivalent to that for Ss with delayed feedback with 
the correct alternative identified and additional information. 

With this form of IF, Ss would have to read the cue and the 
alternatives to find the correct alternative. Thus, It would be 
expected that they responded to all alternatives. Also, the cue 
itself could be used either as a direct associative link between 
the stem and the correct alternative or as a basis for organizing 
the units of the item, i.e., for identifying relationships between 
the stem and the correct alternative and/or the incorrect 
alternatives. Thus, when IF was such that Ss acquired additional 
Information with immediate feedback, retention was equivalent to 
that found under other conditions with delayed feedback. These 
findings suggest that when IF Is presented immediately item-by-item, 
Ss acquire the least information necessary to determine the 
correctness of their previous response. It Is as though their 
response to IF is merely ”1 got that right,” or ”1 got that wrong,” 
and this may be primarily what they are learning. In order to 
improve retention it seeiTis to be necessary that the presentation 
of IF be such that Ss acquire information about the item that is 
relevant to its retention. Apparently, this can be accomplished 
either by delaying the presentation of IF or by manipulating the 
form in which IF is presented. 



The effect of the immediate test conditions upon the delay 
retention effect differed with the form of IF. In Experiment II 
when IF presented a cue, retention following delayed feedback was 
superior to that with immediate feedback only when there was no 
immediate test. However, even when IF directed Ss with immediate 
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feedback to respond to all information present at IF, retention was 
facilitated only when Ss had some immediate practice. Apparently 
when IF was a cue Ss with both immediate and delayed feedback acquire 
more information at IF. However, in order to use this additional 
information to facilitate retention of the correct alternative, it is 
necessary either that IF be delayed or that an immediate test follow 
IF. 

In Experiment i with redundant forms of IF, where additional 
information was the same as on the initial presentation but not 
directly relevant to the recognition tests, the effect upon the delay 
retention effect was different following the immediate recall and 
recognition tests. According to the hypothesis that with delayed IF 
Sjs acquire additional information presented at IF, the following 
interpretation is suggested. The additional redundant information 
acquired with delayed IF could not be utilized directly on the 
recognition test and thus there was no facilitation on later 
retention. However, with minimal cues on the immediate recall test, 
any additional information could be used to facilitate performance 
and later retention was improved. This interpretation is consistent 
with the finding that the overall superiority of delayed IF on the 
immediate tests was greater for the recall test than for the 
recognition test. 

The present findings also indicate that when IF indicates the 
correct alternative and presents additional Information that is not 
the same as that presented initially (Right Wrong-Variable and 
Right-Variable, Experiment I; Right Wrong-Definitions, Experiment II), 
optimal retention requires both a delay of IF and some immediate 
practice. 

it was predicted that the immediate test would provide an 
opportunity for Ss with immediate feedback to acquire additional 
information and thus reduce the superiority of delayed feedback on 
later retention. As indicated above, the effect of the immediate 
test depended upon the information presented at IF and the 
relationship between this and the form of the immediate test. When 
IF was a cue, the effect of the immediate test was to improve retention 
for Ss with immediate feedback and thus decrease the delay retention 
effect as predicted. When IF indicated the correct alternative, 
the effect of the immediate test was to increase the delay retention 
effect. 



According to these findings, it seems that when IF indicates 
the correct alternative, the effect of the immediate test following 
delayed feedback is to increase the Ss ! utilization of additional 
information previously acquired at IF and thus later retention is 
improved. When IF indicates the correct alternative, Ss with 
immediate feedback do not acquire additional information at IF and 
thus the immediate test does not improve later retention. 

Three delay intervals were included to help identify factors 
involved in the effect of 24-hr. delay of feedback. The present 
findings indicate that both the sequence of events and the length 



and lack of experimental activity during the delay interval 
contributed to superior retention with delayed feedback. However, 
the sequence of events contributed more. Thus, it would seem that 
the effect of the longer delay interval is best interpreted as the 
effect of spacing of learning events. The optimal spacing of events, 
such as initial presentation of material, informative feedback, and 
tests, would depend upon a number of variables, some of which have 
been indicated here. 

According to the present findings, the information presented 
at IF in combination with the delay of IF are important in determining 
how Sjs respond to IF and what they acquire at the presentation of IF. 
The information acquired at the presentation of IF and the immediate 
test, as well as the form of this test, determine what Ss retain. 

Experiment III 

Experiment III investigated the question: can effects 

similar to those of delayed feedback be obtained by manipulating 
Ss* responses to feedback. Thus, there were no delay intervals 
compared. The Ss* reactions to IF were manipulated by the form of 
feedback and instructions. Three feedback conditions were compared 
to investigate the effect of: (a) learning the correct alternative; 

(b) learning the correct and the incorrect alternatives; and (c) 
organizing the material. In two of the feedback conditions, IF 
consisted of the entire item with the correct alternative indicated. 

Ss were instructed either (a) to study carefully the correct 
alternative; or (b) to study carefully both the correct and the 
incorrect alternatives. The third feedback condition was the same 
as the Right Wrong-Cue condition in Experiment II. It was expected 
that the cue should promote studying the relationships among the 
units of the item, and thus better organization of the material. 

A second variable investigated the effects of the initial 
presentation with Ss answering each item and the interval preceding 
IF. Thus, in one condition Ss were presented the material, made 
a response, and received feedback. In the other condition, Ss 
received IF but with no initial presentation of the material or 
initial response to the items. As in Experiments I and II there 
were three immediate test conditions and 7-day retention was 
measured with both a recall and a recognition test. 

Method 



Design . Three variables were combined factorial I y: three 

forms of feedback [Instructions Right (l-R); Instructions Right 
Wrong (l-RW); and Right Wrong-Cue (RW-C)]; three immediate test 
conditions [(nothing; recall (Re-1); recognition (Rcg-I)]; and the 
presence or absence of the initial presentation and response 
preceding IF. All Ss had both 7-day retention tests, recall and 
recognition. Subjects were 180 undergraduates,; randomly assigned 
with 10 Ss in each of the 18 groups. 

Learning materia I . Learning material , initial presentation 
for the groups receiving this condition, and the tests were the 
same as in Experiments I and II. For the two instructions groups 
IF was the same as Right Wrong Variable In Experiment I, the stem 
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and all four alternatives In a randomly different position. For 
these two groups Ss were instructed at the presentation of IF (a) to 
study the correct alternative only as long as IF was exposed 
(Instructions Right); or (b) to study the correct and all incorrect 
alternatives while IF was exposed (Instructions Right-Wrong). For 
the Right Wrong-Cue groups feedback and instructions were the same as 
for this condition in Experiment II. 

Procedure . For half of the groups, Ss were presented the 
material, made a response, and received IF for the appropriate 
condition. For these groups the sequence of events was the same as 
for the 20-mi n. delay condition in the previous experiments: the 

series of items was presented, fol lowed immediately by the series of 
IF with the appropriate experimental instructions. For the 
presenTation of the series of items, the temporal sequence was: item 

I (15 sec.); write the answer (15 sec.); item 2 (15 sec.). . . etc. 

For the remaining half of the groups, Ss received IF but with no 
initial presentation of the material or initial response to the Items. 
For these three groups Ss were given the instructions and then the 
series of IF. For all groups the temporal sequence for the series of 
IF was: IF for item I (18 sec.); rest (10 sec.); IF for item 2 

(18 sec.). . ., etc. 

As in Experiments I and II the immediate tests were given 
immediately following the series of IF. All Ss returned seven days 
later for a recall test followed by a recognition test. For all tests 
the temporal sequence was identical to that in Experiments I and II. 

Resu I ts 



Immediate tests . Figure 10 presents the mean correct for each 
group for each of the tests. Table 5 presents the summary of the 
analysis of variance of these data. Performance was superior for Ss 
who received an initial presentation of the material compared with 
those who received IF only. Also, performance on the recognition test 
was superior to that on the recall test. 

Seven-day tests. Figures II and 12 present the mean correct 
for each group for each of the tests. Table 6 presents the summary 
of the analysis of variance of these data. The effect of form of 
feedback was divided into two orthogonal components: F., RW-C vs. 

+ . i-RW); and ? 2 » l-R vs. I-RW. The effect of the Immediate test 
conditions was divided into the same two components as in 
Experiments I and II; T,, the combined tests (Re- I + Rcg-I) vs. no 
test; and T 2 , Re-1 vs. Rcg-I. The interaction between form of IF and 
immediate test conditions was determined on the basis of findings in 
Experiments I and II and was divided Into the following components: 

F| x T,; F, x T 2 ; F 2 x T 2 ; and l-R vs. I-RW for the combined 
i mmed i a te tests ( Reg- 1 + Re- 1 ) . 

As on the immediate tests, performance was superior for Ss who 
received an initial presentation of the material. Also, the overall 
effects of the immediate test conditions and the type of retention 
test were the same as in Experiments I and II. Retention was 
significantly better on the recognition test than on the recall test, 
and it was better following the combined immediate tests than with no 
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A. No Immediate Test 6. Immediate Recall Test C. Immediate Recognition Test 

Fig. 10 Percentage of Errors, 7-day Recall Tests, Incorrect Alternatives from Same Item, Experiment II 
for Delay Conditions, Form of Feedback, and Immediate Test Conditions 






Table 5 

Summary Analysis of Variance, Immediate Tests, Experiment III 



Source 


SS 


df 


MS 


F 


Feedback (F) 


9.83 


2 


4.91 


< 1 


Initial Presentation (IP) 




1 


127.10 


28.43*** 


Test (T) 


3075.47 


1 


3075.47 


688.02*** 


F x IP 


15.18 


2 


7.59 


1.70 


F x T 


3.41 


2 


1.71 


< 1 


IP x T 

(Rcg-I > Re-1, no IP > 


17.25 

IP) 


1 


17.25 


3.86 


F x T x IP 


7.88 


2 


3.94 


< 1 


error 


482.78 


108 


4.47 





***£ < .001 




Immediate test. Following the immediate recognition test 
retention was better than following the immediate recall test; 
and this effect was significantly greater on the 7-day recognition 
test than on the recall test. 

One component of the Interaction between form of feedback 
and Immediate test condition (F x T) was significant. Retention 
for Right Wrong-Cue was superior to that for the combined instruction 
groups ( l-R + l-RW) only when there was no Immediate test. This 
finding Is similar to that In Experiment II, when IF was Right 
Wrong-Cue, in which superior retention with delay of IF occurred only 
with no immediate test. 

Also, one component of the interaction between form of feedback. 
Immediate test condition, and form of retention test (F x T x R) was 
significant. For the combined immediate test conditions (Re- I + Rcg-I) 
retention on the 7-day recognition test was significantly better for 
Instructions-RIght than for Instructlons-Right Wrong; and on the 
recal I test It was superior for Instructlons-Right Wrong. 

Discussion 

The present experiment Investigated the question, can effects 
similar to those of delayed feedback be obtained by manipulating Ss* 
responses to feedback. The effect of form of IF and Immediate tests 
upon retention was similar to the effect of these variables upon the 
delay retention effect in Experiments I and II. That is, in this 
experiment retention was superior for those conditions for which 
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l-RW l-R RW-C 
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Fig. II Mean Correct, Immediate Retention Tests, Experiment III 
for Forms of Feedback and Presence or Absence of Initial 
Presentation (IP) 
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Fig. 12 Mean Correct, 7-day Recall Test, Experiment III for Immediate Test Conditions 
Form of Feedback, and Presence or Absence of Initial Presentation (IP) 
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Fig. 13 Mean Correct, 7-Day Recognition Test, Experiment III for Immediate Test Conditions 
Form of Feedback, and Presence or Absence of Initial Presentation (IP) 



Table 6 



Summary Analysis of Variance, 7-day Retention Tests, Experiment III 



Source 


SS 


df 


MS 


F 


Feedback (F) 


1.55 


2 


0.78 


< 1 


Initial Presentation (IP) 


95.58 


1 


95.58 


14.82*** 


Tests, Immediate (T) 


144.16 


2 


72.08 


11.18*** 


(Rcg-I + Re-1) > none (Ti) 




1 


109.28 


16.94*** 


Rcg-I > Re-1 <T 2 ) 




1 


34.88 


5.41* 


F x T 


46.33 


4 


11.58 


1.80 


RW-C > ( l-RW + l-R) x T ( 




1 


42.03 


6.51* 


F x IP 


18.16 


2 


9.08 


1.41 


F x T x IP 


27.39 


4 


6.85 


1.06 


F, x F 2 x IP 




1 


24.28 


3.76 


error between groups 


1045.59 


162 


6.45 




Retention Test (R) 


6974.00 


1 


6974.00 


3452.48*** 


R x IP 


6.01 


1 


6.01 


2.98 


R x F 


5.17 


2 


2.59 


1.28 


R x T 


40.90 


2 


20.45 


10.12** 


T 2 x R 




1 


39.62 


19.61*** 


F x R x IP 


6.32 


2 


3.16 


1.56 


T x R x IP 


1.84 


2 


0.92 


< 1 


F x T x R 


14.19 


4 


3.55 


1.76 


(Rct-I + Re-1) l-RW vs l-r x 


R 


1 


8.56 


4.23* 


F x T x R x IP 


0.76 


4 


0.19 


< 1 


error within groups 


327 . 44 


162 


2.02 





*p < .05 
**£ < .01 
***£ < .001 
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delayed IF was superior to immediate IF in Experiments I and II. 
When IF was a cue retention was superior only when there was no 
immediate test; and when IF indicated the correct choice retention 
following immediate tests was superior to that with no immediate 
tests. 



Overall retention for Sjs Instructed to study the incorrect 
in addition to the correct alternatives was not superior to that 
for Ss instructed to study the correct alternative only. Instructing 
Ss to respond differently at the presentation of IF did affect their 
retention performance but this effect depends upon the form of the 
retention test as well as the presence of an immediate test. The 
fact that retention on the recognition test was superior when Ss 
had been instructed to study the correct alternative only and on the 
reca I I test when they had been i nstructed to study the i ncorrect as 
well as the correct alternative is consistent with the results of 
Experiment I. In that experiment performance on the recognition test 
was superior when IF presented the correct alternative only and 
on the recall test when IF presented the incorrect in addition to 
the correct alternative. Thus, it appears that Ss did respond to IF 
as they were instructed to. These findings add more support to the 
conclusion that recall of the correct response alternative is 
facilitated when Ss have had an opportunity to identify relations among 
the stem, the correct, and the incorrect alternatives. Also, they 
support the Interpretation that when Ss acquire more information at 
IF, some Immediate practice Is necessary for them to utilize this 
to facilitate retention; and additional information is most 
facilltatlve when retention occurs with minimal cues. 

Overall retention, both Immediate and at seven days, was 
superior when Ss had an Initial presentation of the material and 
wrote their response to that when they received IF only. However, 
the effect of the other variables did not differ with the presence 
or absence of the initial presentation. Thus, It would seem that 
retention Is facilitated by two presentations of the material and 
that this facilitation is Independent of the form of the retention 
test, the Immediate test conditions, or the Ss 1 response to 
feedback. 

As In Experiments I and II, overall retention at seven days 
was better following an Immediate recognition test that after an 
immediate recall test. However, in every experiment facilitation 
of the Immediate recognition test was greater on a later 
recognition test than upon a later reca I I test. Performance on 
the 7-day recall test was just as good following an immediate 
recall test, even though the mean number of correct responses given 
on the immediate recall test was markedly less than that given on 
the immediate recognition test. Apparently, immediate identification 
of a greater number of correct alternatives does not result In 
substantial improvement in later recall of the correct alternatives. 



Conclusions and Recommendations 



Conclusions 

According to the present findings, the following conclusions 
are indicated. The effect of 24-hr. delay of IF differs with the 
form of IF, the presence and form of immediate tests, and the form 
of retention tests. Immediate retention, or acquisition, is 
improved by delayed feedback when retention occurs with minimal 
cues and when IF indicates the correct alternative. When IF 
indicates the correct alternative and presents additional relevant 
information, long-term retention is facilitated when presentation 
of IF is delayed and when there is some immediate practice after 
IF. When IF presents a cue which Ss can use to find the correct 
alternative, optimal retention requires either that presentation 
of IF be delayed or that immediate presentation of IF be followed 
by some immediate practice. 

Superior retention with delayed IF is greater when retention 
occurs with minimal cues. Also, when Ss acquire more information 
at IF some immediate practice is necessary for them to utilize 
this to facilitate retention; and additional information is most 
facilitative when retention occurs with minimal cues. 

These findings are consistent with the interpretation that 
superior retention with 24-hr. delay of feedback is due to factors 
operating at and/or following the presentation of IF and that 
what Ss learn with immediate and delayed feedback actually differs. 
More specifically, they are consistent with the hypothesis that 
with delayed feedback jSs respond to more cues or stimulus aspects 
of IF, thus learning more about the Item; and that when these cues 
can be used in retention, delay improves retention. 

The present findings indicated that the relative superiority 
of delayed feedback on retention could be changed by manipulating 
the form of IF and the Ss* response to IF. When there had been an 
immediate recognition test, the following relationships were 
demonstrated on 7-day retention. When IF indicated the correct 
alternative and was presented in a form different from that of the 
Initial presentation, delayed feedback was superior to immediate. 

When IF was presented in the same form as on the initial presentation, 
which was not the same as on the recognition test, retention with 
delayed feedback decreased and was not superior to Immediate. When 
IF was a cue which could lead to explore all alternatives, 
retention with Immediate feedback improved and did not differ from 
that with delayed feedback. Following an immediate recall test, 
retention with delayed feedback was superior when IF indicated the 
correct alternative whether IF was presented in the same or 
different form from that of the initial presentation. 

The present findings also have implications for understanding 
factors affecting long-term retention of meaningful material. 
Presentation of IF and tests function primarily as learning trials, 
and the effect of learning events upon long-term retention depends 
I upon the conditions of retention. For optimal retention under 

conditions of minimal cues, mere repetition cf the response to the 
stimulus word is not sufficient. Rather, long-term retention is 
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improved when conditions are such that Ss identify relationships 
between the to-be- remembered units and other possible alternatives. 
Perhaps a kind of network is developed in which the correct response 
is integrated with incorrect alternatives; and long-term retention 
is better when there is such a network than when S_s have acquired 
the correct alternative only. According to this interpretation, 
the spacing of learning events and the information presented at 
informative feedback and immediately following it would be important 
In providing opportunity for the development of such a network. 

Optimal retention under conditions In which many cues are present 
Is not so dependent upon organization of the material. However, 
even in this case retention Is facilitated when learning conditions 
are such that some exploration of the material occurs. 

Recommendations 

The results of these experiments have some direct implications 
for educational practices. One general goal of education is that 
the learning acquired be available for long-term retention and 
that it be available under conditions of minimal cues. Thus, 
educational practices should optimize long-term retention with 
conditions more similar to a recall test than to recognition or 
relearning tasks. Following are a few recommendations for 
educational practices, which follow from these findings: 

1. The Immediacy or delay of presentation of information 
about the correct alternative is not so important as what occurs 

at the presentation of this information. If this informative feedback 
Is to improve long-term retention, the first requirement is that it 
be presented under conditions to assure that the student responds to 
the Information provided and not just to determine the correctness 
of his previous response. This may be accomplished by directly 
utilizing some of the* procedures in these experiments or in some 
other way appropriate to the situation. 

2. Retention, especially under conditions of minimal cues, 
will be better If the student learns the correct response in a 
context including more than just that response. This may be 
accomplished by presenting the correct response along with a 
number of other possible alternatives, which are not correct. 

Also, It may be done by presenting information that tells the 
student why it Is correct. 

3. The spacing and form of test or practice activities 
should be determined on the basis of the amount and kind of 
information a student has acquired previously. Although it is not 
indicated In the present experiments. It would be expected that 
the amount and kind of information a student can utilize would vary 
with his level of abilities. Also, this would be expected to 
depend upon his habitual way of responding to information presented. 
Thus, one goal in helping students improve retention is to help 
them to learn to acquire and to use information In addition to 

the correct response. 
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Appendix A 



Sample Item and Different Forms Informative Feedback, Experiments I and tl 



initial Presentation: 



informative Feedback: for Experiment 1 : 

RW+ Right + Wrong - Redundant 
Same Position + letters 



"TO CLEAR FROM BLAME" 
*a. EXCULPATE 

b. LUCUBRATE 

c. LIBRATE 

d. PROPITIATE 



RW Right + Wrong - Variable 

Randomly different position - 
no letters 



"TO CLEAR FROM BLAME" 

a. EXCULPATE 

b. LUCUBRATE 

c. LIBRATE 

d. PROPITIATE 



R+ Right only - Redundant 
Same Position + letter 



"TO CLEAR FROM BLAME" 
*a. EXCULPATE 



R Right only - Variable 

Randomly different position 
no letter 



"TO CLEAR FROM BLAME" 




"TO CLEAR FROM BLAME" 


PROPITIATE 






LIBRATE 




* EXCULPATE 


* EXCULPATE 






LUCUBRATE 







Informative Feedback: for Experiment II : 



RW-D Right + Wrong - Definitions RW-C Right + Wrong - Cue 



"TO CLEAR FROM BLAME" 




"TO CLEAR FROM BLAME" 


LIBRATE (vibrate) 




LUCUBRATE 


PROPITIATE (pacify) 




EXCULPATE 


LUCUBRATE (study laboriously 


> 


PROPITIATE 


* EXCULPATE \ 




LIBRATE 






(EX = OUT; CULP = GUILT, 






AS IN CULPRIT) 



Tests, Experiments I and II: 



Recognition; 



Reca 1 1 : 







Hiiai 



"TO CLEAR FROM BLAME" 






"TO CLEAR FROM BLAME" 
PROPITIATE 
EXCULPATE 
LUCUBRATE 
LIBRATE 







\ 
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Appendix B. Temporal Sequence of Events for the Three Delay Conditions, Experiments I 
0-mln. Delay: 

Sess Ion I : 
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7 days after Session 2: Recall Test; Recognition Test 

All Three Delay Conditio ns; Immediate and 7-day Recall and Recognition Tests 

Item I Write Answer Item 2 ... 

S3 sec. 15 sec. 15 sec. 

NOTE: In all conditions, Ss had the same number, type, and length of presentation of all material. 



